You can delete the active version of a document, or any non-active version. When you delete the active version of a document, all its non-active versions are deleted with it. Deleting a non-active version is the same as deleting a document that does not have versions.

Just like deleting regular documents, to delete a document and all its versions, your user settings Document - Modify and Document - Delete must both be on, and you must have your access control permission, Delete, turned on for the document(s) you want to delete.